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Abstract 

We consider a stochastic control problem where the set of controls 
is not necessarily convex and the system is governed by a nonlinear 
backward stochastic differential equation. We establish necessary as 
well as sufficient conditions of optimality for two models. The first 
concerns the strict (classical) controls. The second is an extension of 
the first to relaxed controls, who are a measure valued processes. 
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1 Introduction 

In this paper we study a stochastic control problem where the system is 
governed by a nonlinear backward stochastic differential equation (BSDE for 
short) of the type 

dyl = h{t,ylzl,Vt)dt + zldWu 
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where b is given function, ^ is the terminal data and W = (Wt)^^^ is a 
standard (i-dimcnsional Brownian motion, defined on a filtered probability 
space JF, {J^t)t>Q i 'P) satisfying the usual conditions. The control variable 
V = (vt) , called strict (classical) control, is an jF^-adapted process with values 
in some set U of R*^. We denote by U the class of all strict controls. 
The criteria to be minimized, over the set U, has the form 



J{v)^E 



T 



g{yl)+ / hit,yizivt)dt 







where g and h are given maps, and {y^, z^) is the trajectory of the system 
controlled by v. 

A control u & U is called optimal if it satisfies 

JH = infJ(^;). 

Stochastic control problems for the backward and forward-backward sys- 
tems have been studied by many authors. The first contribution of control 
problems of forward-backward systems is made by Peng [30] , he obtained the 
maximum principle with the control domain being convex. Xu [34] estab- 
lished the maximum principle for this kind of problem in the case where the 
control domain is not necessary convex, with uncontrolled diffusion coeffi- 
cient and a restricted functional cost. The work of Peng [30] (convex control 
domain) is generalized by Wu [33], where the system is governed by a fully 
coupled forward-backward stochastic differential equation. Shi and Wu [32] 
extend the result of Xu [34] to the fully coupled forward-backward systems, 
with convex control domain and uncontrolled diffusion coefficient. Ji and 
Zhou [22] use the Ekeland variational principle and establish a maximum 
principle of controlled forward-backward systems, while the forward state is 
constrained in a convex set at the terminal time, and apply the result to state 
constrained stochastic linear-quadratic control models and a recursive util- 
ity optimization problem are investigated. All the cited previous works on 
stochastic control of forward-backward systems are obtained by introducing 
two adjoint equations. In the recent works on the subject, Bahlali and Labed 
[3] and Bahlali [6] introduce three adjoint equations to establish necessary as 
well as sufficient optimality conditions. In [3] the authors establish the re- 
sults in the case where the control domain being nonconvex and uncontrolled 
diffusion coefficient. The results of [6], are obtained while the control domain 
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is convex and with controlled diffusion coefficient, moreover the author apply 
his theory to solve the financial model of cash flow valuation. 

On the other hand, stochastic maximum principle of backward systems 
was studied by El-Karoui et al [14] , where the linear case is solved and some 
applications in finance are treated. Dokuchaev and Zhou [9] established 
necessary as well as sufficient optimality conditions, where the control domain 
is not convex. 

Our objective in this paper is to establish necessary as well as sufficient 
optimality conditions, of the Pontryagin maximum principle type, for two 
models. 

Firstly, we derive necessary as well as sufficient optimality conditions for 
strict controls. Since the set of strict controls is nonconvex, the classical way 
to use, is the spike variation method. More precisely, if u is an optimal strict 
control and v is arbitrary, then with a sufficiently small 6' > 0, we define a 
perturbed control as follows 



We then derive the variational equation from the state equation, and the 
variational inequality from the fact that 



The major difficulty in doing this is that the state of a backward system 
and the functional cost depends on two variables yt and Zt- Then, we can't 
derive directly the variational inequality, because Zt is hard to handle, there 
is no convenient pointwise (in t) estimation for it, as opposed to the first vari- 
able Uf. To overcome this difficulty, we introduce a new method which consist 
to transform the initial control problem to a restricted problem without in- 
tegral cost, by adding an unidimensional BSDE. We establish then necessary 
optimality conditions for the restricted control problem and by an adequate 
transformation on the adjoint process and the adjoint equation associated 
with the restricted problem, we reformulate necessary optimality conditions 
for the initial control problem. 

To achieve this part of the paper, we study when these necessary opti- 
mality conditions becomes sufficient. 

The second main result in this paper concerns necessary as well as suf- 
ficient optimality conditions for relaxed controls. In the relaxed model, the 





otherwise. 



< J {u^) -J{u). 
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controller chooses at time t a probability measure qt [da) on the control set 
U, rather than an element Vf of U. The system is then governed by the BSDE 



(^Vt = lu^ ^t, «) Qt (da) dt + z'^dWt, 

The criteria to be minimized, over the set TZ of relaxed controls, has the 
form ^ 

J{q)=¥. g iyl) + [ [ h{t, yl zl a) qt {da) dt 
Jo Ju 

A control fj, E TZ is called optimal if it satisfies 

J{^i) = miJ{q). 

The relaxed control problem is an extension of the previous model of 
strict controls. Indeed, if qt (da) = S^^ (da) is a Dirac measure concentrated 
at a single point Vt, then we get a strict control problem as a particular case 
of the relaxed one. 

By using the Ekeland's variational principle, we are able to estabhsh 
necessary optimality conditions for near optimal strict controls converging in 
some sense to the relaxed optimal control, by the so called chattering lemma. 
The relaxed necessary optimality conditions arc then derived by using some 
stability properties of the trajectories and the adjoint process with respect 
to the control variable. 

We note that necessary optimality conditions for relaxed controls, where 
the systems arc governed by a stochastic differential equation, were studied 
by Mezerdi and Bahlali [27], Bahlali, Djehiche and Mezerdi [4] . 

The paper is organized as follows. In Section 2, we formulate the problem 
and give the various assumptions used throughout the paper. Section 3 is 
devoted to restrict the initial control problem to a problem without integral 
cost and we derive a restricted necessary optimality conditions. In Section 
4, we give our first main result, the necessary optimality conditions for the 
initial control problem and under additional hypothesis, we prove that these 
conditions becomes sufficient. Finally, in the last Section, we give necessary 
optimality conditions for near optimal controls and from this we derive our 
second main result in this paper, necessary as well as sufficient optimality 
conditions for relaxed controls. 
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Along this paper, we denote by C some positive constant, Ainxd ^ 
the space of n x real matrix and Mn-^n (^) linear space of vectors 
M = (Ml, Md) where Mj e Mnxn {^)- We use the standard calculus of 
inner and matrix product. 



2 Formulation of the problem 

Let (Q, {y-'t)t>o ) ^) be a probability space equipped with a filtration sat- 
isfying the usual conditions, on which a d- dimensional Brownian motion 
W = (VFf)f>o is defined. We assume that {J^t) is the V- augmentation of 
the natural filtration of (VFt)t>o • 

Let T be a strictly positive real number and U a non empty subset of R'^. 

Definition 1 An admissible control is an Tt- adapted process with values in 
U such that 



E 



sup \Vt\ 
te[o,T] 



< oo. 



We denote by U the set of all admissible controls. 
For any v &U, we consider the following BSDE 

r dy^^b{t,y^,zlvt)dt + z^dWt, 

where 

b : [0, T] X X Mnxd (R) X [/ — > R", 
and ^ is an n-dimensional J^T-measurable random variable such that 

E|e|'<oo. 

The expected cost is defined from U into R by 



(1) 



J{v)=E 



9iyo)+ rh{t,ylzlvt)dt 
Jo 



(2) 



where 



9 ■ 



h : [0,T] X R" X Mnxdi^) X U 
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A control u eU is called optimal, if that solves 

J (u) = inf J (v) . (3) 

Our goal is to establish necessary as well as sufficient optimality conditions 
for controls in the form of stochastic maximum principle. 

The following assumptions will be in force throughout this paper 

The functions b, g and h are continuous in (y, v) , they are (4) 
differentiable with respect to (y, z) , and they derivatives 
by, bzi Qy, hy and are continuous in (y, z, v) and uniformly bounded. 
b and h are bounded byC(l + |y| + |T;|) and bounded in z. 

Under the above hypothesis, for every v E U, equation (1) has a unique 
strong (jFj) ^-adapted solution and the functional cost J is well defined from 
U into M. 



3 Problem with restricted cost 

Since the function h of the cost depend explicitly on Zt, we can't treat our 
problem directly. Thus, let us in this section restrict the initial control 
problem {(1) , (2) , (3)} to a problem without integral cost. For this end, 
consider the following unidimensional BSDE 

r dx!l = h {t, zl vt) dt + k^,dWt, 

where k" is an (1 x d) matrix, {y^, z^) is the solution of equation (1) and r) 
is an one-dimensional J?-T-measurable random variable such that 

E \r]f < oo. 

The above equation admits a unique strong {J-'t)^- adapted solution. 
We put 
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and consider now the following {n + 1) -dimensional BSDE 



dyt = b {t, yt, Zt, Vt) dt + ZtdWt, 



V 



(5) 



where the functions b is defined from [0, T] x M"+^ x M(n+i)xd {^) x U into 

M"+^ by 

b{t,ylz];,vt) 



and it is a (n + 1) X d real matrix given by 



/ V V \ 

^21 ^22 ••■ 



nl ^n2 ••• ^nd 



From (4), b is uniformly Lipschitz in {yt,Zt), then equation (1) admits a 
unique strong solution {yt,Zt) adapted to the filtration {J-t)f 



Define now the function g from M.^'^^ into M by 

9{yt) = 9{yt)-xl 

and the new functional cost from U into M by 

J{v)=¥.[g{yo)]+E[7^]. 

It's easy to see that 



(6) 



J(v)^J(v) 



Consequently, it's sufficient to minimize the restricted cost J over U. If 
e W is an optimal solution, that is 



JH = inf J(^). 



(7) 



Prom this transformation, we have reduce our initial problem {(1) , (2) , (3)} 
to a new problem without integral cost. We can now study the restricted 
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problem {(5) , (6) , (7)} by using a classical way of spike variation method. 
We establish necessary optimality conditions for a restricted problem and by 
an adequate transformation on the adjoint process and the adjoint equation 
associated with the restricted problem, we reformulate necessary optimality 
conditions for the initial control problem {(1) , (2) , (3)}. 



3.1 Preliminary results 



Suppose that m e W is an optimal control and denote by {yt, Zt) the solution of 
(5) corresponding to u. Introduce the following perturbation (spike variation) 
of the optimal control u 



V \lt e[T,T + 9], 

Ut otherwise. 



(8) 



where < r < T is fixed, ^ > is sufficiently small and v is an arbitrary 
^t-measurable random variable with values in U such that E [|^;|^] < oo. 

The control is admissible and let (yf , if) be the solution of (5) associ- 
ated with u^. 

Since u is optimal, the variational inequality will be derived from the fact 
that _ _ 

< J {u^) -J{u). (9) 

For this end, we need the following lemmas. 



Lemma 2 Under assum,ptions (4), we have 



E 



sup \yt -yt\ 

te[o,T] 



< C9^, 



E 



\^t - ^t\ dt<C9^ 



(10) 



Proof. We have 



d {yt - yt) 



dt 



[ {y'T-yr)^ 0. 



6(t,yf,if,M?) -6(t,yi,if,n?) 

b{t,yt,zt,u^t) -'b{t,yt,zt,u^t)] dt 
b{t,yt,zt,u^t) -b{t,yt,zt,ut)^ dt 
+ (if - it) dWt, 
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Put 



— h ~ ^t, 



and 



{t, y/, Z^) = / by {t, y, + X{g-y,),Zt + X{^,- Zt) , «a r/dA (12) 

Jo 

+ f h{t,yt + X{g-yt),zt + X{^,-zt),ut)Z',dX 
Jo 

+ b{t,yt,zt,uf) -b{t,yt,zt,ut) . 

dYf = cp<^ {tX,Z!) dt + ZfdWt, 



Then 



Y^ = 0. 



(13) 



The above equation is a hnear BSDE with bounded coefficients and with 
terminal condition — 0. Then by applying a priori estimates (see Briand 
et al [8, Proposition 3.2, Page 7]), we get 

2 

|/(i,0,0)|cii . 











E 


sup + / \Z^\^dt 


< CE 


f 




t€[0,T] Jo 




Jo 



From (12), we get 





nT 






E 


sup + / \Zf\^dt 
te[o,T\ Jo 


< CE 


Jo 



By the definition of , we have 



E 



te[o,r] 











< CE 


r 


Jo 








< CE 


sup 



te[o,T] 



6 {t,yt,zt,uf) -b{t,yt,zt,ut) 



b{t,yt,zt,v) -b{t,yt,zt,ut) 



b{t,yt,zt,v) -b{t,yt,zt,ut) 



dt 



dt 



-+0 



dt 



By (4), b is with linear growth with respect to {y,v) and bounded in z, 
then b satisfy the same properties, and we get 









E 


sup / \zffdt 
te[o,T] Jo 





The lemma is proved. 
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3.2 Necessary optimality conditions for restricted prob- 
lem 

We can now state necessary optimality conditions for a restricted control 
problem {(5), (6), (7)}. 

Theorem 3 (necessary optimality conditions for restricted problem) Let {u,y,T) 
be an optimal solution of the restricted control problem {(5) , (6) , (7)}. Then 
there exists a unique adapted process 

peC ([0,T];M-+i), 

which is solution of the following forward stochastic differential equation 

-dpt = Hy (t, yt, zt,pt, ut) dt + {t, yt, Zt,pt, Ut) dWt, 
Po = 9y {yo) , 

such that 

H {t,yt,zt,pt,ut) ^ maxH {t,yt,zt,pt,v) ; a.e , a.s, (15) 

where the Hamiltonian H is defined from [0,7] x R"+^ x Al(n+i)xci (1^) x 
X U into M hy 

H {t,yt,zt,pt,ut) = b{t,yt,zt,ut)pt. 
Proof. For simplicit, we put 

At^{t,yt + X{yt-yt),Zt + X{^,-z,),u'l). 
Since u minimizes the cost J over U, then ■ 



< J {u^) - J (u) 
<E[g{y',)-giyo)] 

<e[ [y, + X {y^^ - y,)] [y^ - y^) d\ 

Jo 

< E [gy {yo) {y'o -yo)]+E [gy {yo + X {y'o - m)) - 9y {Vo)] {Vo - Vo) dX. 
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We remark from (14) that 



Po = 9y iVo) ■ 



Then 



0<E[poK-yo)] +Ef [gy {yo + X {fo - yo)) -9y{yo)] K - yo) dX. 

Jo 

(16) 



By applying Ito's formula to pt (yf — yt) , we get 



E[po{y',-yo)] =E 



'■T fl 




10 JO 

cT pi 



+ E 



Then (16) becomes 



< E 



by (At) - by {t, yt, zt, ut) {yt - yt) PtdXdt 

+ e[ [ \b, {A't) - \ (t, yt, Zt, Ut)] {zt - ^t) PtdXdt 
Jo Jo ^ 

bit, yt, Zt, Ut) -b{t, yt, Zt, Ut) Ptdt. 



dt 



H [t, yt, zt,pt, Ut) - H [t, yt, Zt,Pt, uf) 
E / [gy {yo + X{y^o-yo)) -9y (yo)] {yt - yf) dX 

Jo 

by (At) - by {t, yt,zt, Ut) {yt - yt) PtdXdt 

+ e[ I (At^) - {t, yt, Zt, Ut)] {zt - ^t) PtdXdt 
Jo Jo ^ 



+ E 




'0 ^0 



Let us show that 



E 




Jo 



and 



E 




^0 



(17) 



by (A?) - by {t, yt, Zt, Ut) {yt - yf) PtdXdt < CQ^I\ (18) 



6, (A^) - b, {t, yt, Zt, Ut)] {zt - ^t) PtdXdt < Cd^l\ (19) 
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Indeed, by using the Cauchy-Schwartz inequality to term in the left hand 
side of (19), we get 



T rl 



E 




^0 



< E 




(At ) - {t, yt, zt, ut) {zt - z^) ptdXdt 

b, (At) - bz {t, yt, Zt, Ut)] pt ^ dXdt] ( E / \lt-^t\'dt 



T rl 



^0 



9 \ 1/2 / \ 1/2 



By (11), we obtain 



E 




Jo 



<ce{E 



bz (Aj) - bz {t, yt, Zt, Ut) {zt - z^) ptdXdt 
bz {A^ - bz {t,yt,zt,ut) 



T rl 




'0 ^0 

By the definition of u^ , we have 



Pt 



N 1/2 

dXdt I 



Jo Jo 

<ce(E 



bz (Af) - bz {t, yt, Zt, Ut) {zt - if) ptdXdt 



T JO 



dXdt 



1/2 



bz {AD - bz {t,yt,zt,ut)pt 
Since by is bounded, we get 

E^ ^ [ft, (A,^) - bz {t, yt, Zt, Ut)] (if - it) PtdXdt 

/ rr+e \ 1/2 

<C9lj E\p\'^dt] . 
Since p e jC^ ([0, T] ; W+'^) , we obtain 



T rl 



E 




^0 



bz (Af ) - bz {t, yt, Zt, Ut) (if - zt) ptdXdt 



-r+e X 1/2 

< ic I dt] ce^ cd^i'^. 



Relation (19) is proved. 
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(18) is proved by the same method and by using (10) and the fact that 



by is bounded. 



Now, by (17), (18) and (19) we get 



< E 



H {t,yt,zt,pt,ut) - H {t,yt,zt,pt,u1) 



dt 



+ E / [9y {yo + ^{fo-yo)) - 9y (l/o)] " l/o) d\ 

Jo 

+ ce^/\ 

By applying the Cauchy-Schwartz inequality to the second term in the 
right hand side of the above inequality, we get 



< E 



H {t, yt, zt,pt, ut) - H (t, yt, Zt,pt, u^) 
+ ^ E \gy {yo + A (y^ - yo)) - Qy (yo) f ciAj (e g - ^0!') 



dt 

1/2 



1/2 



By (10), we deduce 
< E 



H {t, yt, zt,pt, Ut) - H {t, yt, Zt,Pt, u^) 
+ Ce \gy {yo + X{f,-yo))-gy {yo)\' dX 



dt 



1/2 



From the definition of ul, we have 



< E 



H {t,yt,zt,pt,ut) - H {t,yt,zt,pt,v) 



dt 



+ ce \gy {yo + \{^o-yo))-9y (^0) f dX^ 



1/2 
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Dividing by ^, we get 



0<-E 



dt 

1/2 



(20) 



H {t, yt, zt, pt, ut) - H {t, yt, Zt,pt, v) 
+ C (^J\ \gy {yo + A (y^ - y^)) - gy (yo) f dX^ 

Since 'gy is continuous and bounded, then by (10) and the dominated 
convergence theorem, we have 

limC^' (e \gy {yo + X {y", - yo)) - gy (yo) f) dX = 0. 
Then, by taking the hmit as ^ ^ in (20), we obtain 



< hm-E 
This implies that 



H {t,yt,zt,pt,ut) - H {t,yt,zt,pt,v) \ dt. 



< E 



H {T,yr,Zr,Pr,Ur) - H {t, yr, Z^, Pr, v)^ , dr - Q.e. 



Now, let a e J7 be a deterministic element and F be an arbitrary element 
of the (7-algebra and set 

wt = alp + Utln-F- 
It is obvious that w is an admissible control. 

Since < r < T, then for every bounded ^/-valued, JF^- measurable ran- 
dom variable v such that E|vp < +oo, we get 



< E 



, dt — a.e, 



H {t,yuzt,pt,ut) - H {t,yt,zt,pt,v) 

Applying the above inequality with w, we get 

< IKllpiH {t,yt,zt,pt,ut) - H {t,yt,zt,pt,a))], VF e ^t, 

which implies that 

< E[H {t, yt, zt,pt, Ut) - H {t, yt, Zt,pt, a) / Tt]. 

The quantity inside the conditional expectation is jF^-measurable, and 
thus the result follows immediately This prove theorem 3. 
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4 Necessary and sufficient optimality condi- 
tions for strict controls 



Starting from the results of the last section, we can now reformulate the 
restricted necessary optimality conditions given by theorem 3, and state nec- 
essary as well as sufficient optimality conditions for the initial control problem 
{(1),(2),(3)}. 

4.1 Necessary optimality conditions 

Theorem 4 (necessary optimality conditions for strict controls) Let {u, y", z") 
be an optimal solution of the initial control problem {(1) , (2) , (3)}. Then 
there exists a unique adapted processes 

e£'([o,r];R"), 

which are solution of the following forward stochastic differential equation 
-dp^ = Hy (t, y^, zt, , ut) dt + it, yX, z^, , Ut) dWt, . 



such that 



H (t, vX, z't^p'l, ut) = max// {t, y^, z't.p't, v) ; a.e , a.s, (22) 



where the Hamiltonian H is defined from [0, T] x M" x Mnxd x x [/ 
into M hy 

H (t, y, z,p, v) = pb (t, y, z,v)-h (t, y, z, v) . 

Proof. We put 

From the definition of i/, p, b and z, we have 

H {t, yt, zt,pt, Ut) = H {t, yl z^pl Ut) , (23) 

and from the adjoint equation (14), we can easily deduce (21). Finally (22) 
is derived immediately from (23) and (15). ■ 
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4.2 Sufficient optimality conditions 

Theorem 5 (Sufficient optimality conditions for strict controls). If we as- 
sume that, U is convex and for every v & U and for all t e [0,7"]; the 

function g is convex and {yt,Zt,Vt) — > H (t,yt, Zt,ptjVt) is concave. Then u 
is an optimal control of the problem {(1) , (2) , (3)} if it satisfies (22) . 

Proof. Let u be an arbitrary admissible control (candidate to be optimal) 
and (y", z^) the solution of (1) associated with u. For any admissible control 
V, with associated trajectory {y'^,z^), we have 

J{v)-J{u)^E[g{y^,)-g{y^)] 



+ E [h (t, yl zl vt) - h (t, i/r, ut)] dt. 
Jo 

Since g is convex, then 

9{yo)-9{yo)>9y{yo) iVo-Vo)- 

Then 

J{v)-J{u)>E[gy{y^) {y'o ' Vo)] 

+ e[ [h {t, yl zl Vt) - h {t, yr, zl Ut)] dt. 
Jo 

We remark from (21) that 

Po = 9y (Vo) ■ 

Then, we have 

J{v)-J{u)>E[p^,iy^,-y^)] 

+ e[ [h {t, yl zl Vt) - h {t, y^, zl Ut)] dt. 
Jo 

By applying Ito's formula to {y^ — y^), we obtain 
J{v)-J (u) 

> E [Hy {t, i/r, z:,pl Ut) {yl - i/r) + {t, i/r, z:,pl Ut) {z^ - zt)] dt 
Jo 

+ E / [H {t, yr, - H {t, yl zlpl Vt)] dt. 

Jo 
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Since H is concave in {y, z, u), then 

H {t, zlplvt)-H {t, yl z]^,p^, 

<Hy{t,y:,z:,plut) (y't-y:) 

+ {t, y^, z]:,pl ut) {zl - zX) + {t, y^, z^,pl Ut) {vt - Ut) . 

Or equivalently 

Hvit,yt,z^,p'l,ut) {ut-vt) 
< H {t, y^, z^, pl Ut) - H {t, yl z^ , p^ v^) 

+ Hy {t, yl zlp^, Ut) - l/r) + {t, yl z^p^, Ut) {z^ - z^) . 
Then, we get 

J{v)-J{u)>e[ {t, yl zl plut) {ut - Vt) dt. (24) 

We know that H {t, y^, z^jp"^, .) is concave, then —H (t, y^, z^,p^, .) is con- 
vex from U into R. Furthermore U is convex and —H {t, y^, z^,p^, .) is con- 
tinuous, Gateaux-differentiable, with differential continuous, then from the 
convex optimization principle (see Ekeland-Temam [11, prop 2.1, page 35]), 
we have 

-H {t, yl z^pl Ut) = inf -// {t, y^, z^pl Vt) ^ -H, {t, yl z^,pl Ut) {vt - Ut) 

vteu 

Or equivalently 

H {t, y^, z^pl Ut) = max/7 (t, y^, z^pl Vt) ^ {t, y^, z^p^ Ut) {ut - Vt) > 0. 
vteu 

Then from the necessary condition of optimahty (22), we deduce that 

H^it^y^z^p^ut) {ut-vt)>0. 
And from (24), we have 

J{v) - J{u)> 0. 
The theorem is proved. ■ 
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5 The relaxed model 



In this section, we generalize the results of the above section to a relaxed 
control problem. The idea for relaxed the strict control problem defined 
above is to embed the set U of strict controls into a wider class which gives 
a more suitable topological structure. In the relaxed model, the [/-valued 
process v is replaced by a P (f/)-valued process q, where P (U) denotes the 
space of probability measure on U equipped with the topology of stable 
convergence. 

Let V the set of positive random measures on [0, T] x [/ whose projection 
on [0, T] coincide with the Lebesgue measure dt. Equipped with the topol- 
ogy of stable convergence of measures, y is a compact metrizable space. 
The stable convergence is required for bounded measurable functions / {t, a) 
such that for each fixed t G [0,T], h{t,.) is continuous. The space V is 
equipped with its Borel cr-field, which is the smallest cr-field such that the 

mapping q i — > J f (s, a) q {ds, da) are measurable for any bounded measur- 
able function /, continuous with respect to a (Instead of functions bounded 
and continuous with respect to the pair {t, a) for the weak topology). 

For more details, see Jacod-Memin [18, page 629-630] and El Karoui et 
al [9, Page 4-5] . 

Definition 6 A relaxed control {qt)^ is a F (U) -valued process, progressively 
measurable with respect to {J-t)t o-i^d such that for each t, l]o,t]-? is Tt- 

measurable. 

We denote by TZ the set of all relaxed controls. 



Every relaxed control q may be desintegrated as q {dt, da) = q {t, da) dt = 
qt (da) dt, where qt (da) is a progressively measurable process with value in 
the set of probability measures P(f/). 

The set U is embedded into the set 71 of relaxed process by the mapping 

f : V & U\ — >fv {dt, da) — Sy^{da)dt e TZ 
where 5y is the atomic measure concentrated at a single point v. 

For more details on relaxed controls, see [2] , [4] , [5] , [12] , [16] , [26] , [27] . 
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For any q e TZ, we consider the following relaxed BSDE 



dVt = lu^ ^t: «) Qt (da) dt + z^dWt, 



(25) 



The expected cost associated to a relaxed control q is defined as follows 



9{yl) 



JU 



h{t,y^,z^,a)qt{da) dt 



(26) 



Our objective is to minimize the functional over TZ. li /i & TZ is an 
optimal relaxed control, that is 



J (/.) = inf J (g) . 

q&TZ 



(27) 



Throughout this section we suppose moreover that 

U is compact, 

b and h are bounded, 

by,hy, bz and hz are Lipschitz continuous in z. 
Remcirk 7 // we put 

b {t, yl z1, qt)= b {t, yl, zf, a) qt {da) , 
Ju 



h it, yl z1, qt)= h (t, yl, z^ a) qt (da) , 



u 



then equation (25) becomes 



dy? = b (t, yf, zl, qt) dt + zldWt, 



with a functional cost given by 



J (g) = E 



aiyl) + / h(t,y1,z1,qt)dt 



(28) 



Hence by introducing relaxed controls, we have replaced U by a larger 
space P(?7). We have gained the advantage that P (f/) is both compact and 
convex, the new drift and the integral coefficient of J are linear in q. 
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On the other hand, the coefficients b (defined above) check the same as- 
sumptions as b. Then, under assumptions (4), b is uniformly Lipschitz and 
with linear growth. Then, by classical results on BSDEs (The Pardoux-Peng 
theorem, see : Pardoux-Peng [28]^, for every q E 71, equation (25) has a 
unique solution. 

Moreover, It is easy to see that h checks the same assumptions as h. 
Then, the functional cost J is well defined from TZ into M. 

Remcirk 8 If qt — is an atomic measure concentrated at a single point 
Vt, then for each i e [0, T] u;e have 

b {t, y!, zl, a) qt (da) = / b{t, yl zf, a) 5^^ {da) = b {t, yl zl Vt) , 

Ju 

h {t, yl zl a) qt (da) = h{t, yl zl a) {da) = h {t, yl zl Vt) . 

Ju 

In this case {y^,z^) = {y'",z'"), J {v) = J {q) and we get an ordinary 
admissible control problem. So the problem of strict controls defined in the 
section 2 is a particular case of the problem of relaxed one. 

5.1 Approximation of trajectories 

The next lemma, known as the Chattering Lemma, tells us that any relaxed 
control is a stable limit of a sequence of strict controls. This lemma was first 
proved for deterministic measures and then extended to random measures in 
[12] and [16]. 

Lemma 9 (Chattering Lemma). Let qt be a predictable process with values 
in the space of probability measures on U. Then there exists a sequence of 
predictable processes {u^)^ ^^^^ values in U such that 

dtq^ {da) — dt6u" {da) — > dtqt {da) stably, V — a.s. (29) 

* n >oo 

Proof. See El Karoui et al [12] . ■ 

Lemma 10 Let q be a relaxed control and {u'^)^ be a sequence of strict con- 
trols such that (29) holds. Then for any bounded function f : [0,T] xU -^M., 
measurable in t and continuous in a, we have 



L 



f f {t, a) 5„n {da) [f{t, a) qt {da) . (30) 

u Ju 
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Proof. By the Chattering lemma and the definition of the stable convergence 
(see Jacod-Memin [21, definition 1.1, page 529], we have 

/ / (t, a) 5u" (da) dt — > / f {t, a) qt (da) dt. 
Ju * Jo Ju 



Put 

It's clear that 

fT 



9 {s, a) = l[o,t] (s) / (s, a) . 



/ I g{s, a) (da) ds — >• / 9 {s, a) Qs (da) ds. 
Jo Ju " »°°7o Ju 



Then 



nf (s, a) Sun (da) ds — > / / / (s, a) Qs (da) ds. 
n—*oo 

The set {{s,t) ; < s <t <T} generate B[o,t]- Then VS e B[o,t], we 
have 

/ f{s, a) Sun (da) ds — > / f (s, a) (da) ds. 
JbJu JbJu 

This imphes that 

/ / (s, a) Sun {da) — > f {s, a) Qs {da) , dt - a.e. 

Ju n— »oo Jjj 

The lemma is proved. ■ 

The next lemma gives the stability of the controlled stochastic differential 
equation with respect to the control variable. 

Lemma 11 Let qt & TZ be a relaxed control and {y'^,z'^) the corresponding 
trajectory. Then there exists a sequence C U such that 



limE 

n— »oo 



sup Ivt -Vtl 
te[o,T] 



0, 



(31) 



limE 

n— >oo 



\z^-z!fdt^O, 



(32) 

lim J (m") = J{q), (33) 
where {y^,z"-) denotes the solution of equation (1) associated with u^. 
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Proof. We have 

d {y^ - yi) = [b {t, y^, z^, <) - b {t, yl zf, <)] dt 



+ 



b (t, yl zl u'l)- b {t, yl zl a) qt (da) 
Ju 



dt 



+ {z^ - z!) dW, 



Put 



yr = y't - yl 

'yn _ n q 
— h ~ 



and 



^" (t, Z^) = b {t, yl zl u'l) - fb {t, yl zl a) qt {da) (34) 

Ju 

+ [\y {t, yl + A (yr - yl) ,zl + \ {z^ - z^) , <) Y.^dX 
Jo 

+ [\, {t, yl + A (yr - yl) ,zl + X {z^ - zl) , <) ZfdA. 

^0 



Then 



Yf = 0. 



(35) 



The above equation is a hnear BSDE with bounded coefficients and with 
terminal condition Y^p — 0, then by applying a priori estimates (see Briand 
et al [8]), we get 



E 









sup / \Z^fdt 


< CE 




te[o,T] Jo 




Jo 



Prom (34), we get 



E 



sup \Y^\^+ I \Z^\^dt 
te[o,T] 



(36) 



< CE 



T 



' b {t, yl zl a) Su^ (da) - b{t, yl zl a) qt (da) 
u Ju 



dt. 
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By (30) and the dominated convergence theorem, the term in the right 
hand side of the above inequahty tends to zero as n tends to infinity. This 
prove (31) and (32). 



Let us prove (33) 

Since g and h are Lipshitz continuous in {y, z), then by using the Cauchy- 
Schwartz inequahty, we have 



1/2 . 

+ C E 



\J{u^)-J{q)\ 

< c (e - ) + ^ (/ E \yt - y!\' ds 

b {t, v!, zl, a) (5„n {da) - h{t, yl zf, a) qt {d 

T Ju 



z^-zlfdt 



1/2 



+ C E 



2 \ V2 

dt 



From (31) and (32) the first, the second and the third terms in the right 
hand side converge to zero, and by (30) and the dominated convergence 
theorem, the fourth term in the right hand side tends to zero. ■ 

Remark 12 As a consequence, it is easy to see that the strict and relaxed 
optimal control problems have the same value function. 



5.2 necessary optimality conditions for near controls 

In this section we derive necessary optimahty conditions for near optimal 
controls. This result is based on Ekeland's variational principle which is 
given by the following. 

Lemma 13 (Ekeland's variational principle). Let {E,d) be a complete met- 
ric space and f : E — > M. be lower- semicontinuous and bounded from below. 
Given e > Q, suppose u'^ E E satisfies f (u^) < inf (/) + s. Then for any 
A > 0, there exists v & E such that 



1. f{v)<f {u^) . 

2. d{u',v) < X. 

3. fiv)<f (w) + j-d{v,w) , V w 7^ V. 
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Proof. See Ekeland [10] . ■ 

To apply Ekeland's variational principle, wc have to endow the set U of 
strict controls with an appropriate metric. For any u,v eU, we set 

d{u,v)^V® dt {{u, t) e n X [0,T] , u {t, u;)y^v {t, u;)} , 

where V ® dt is the product measure of V with the Lebesgue measure dt. 

Let us summarize some of the properties satisfied by d. 

Lemma 14 1. {U, d) is a complete metric space. 
2. The cost functional J is continuous from U into M. 

Proof. Sec Mczcrdi [25] . ■ 

Now let /i G 7^ be an optimal relaxed control and denote by (|/^, z^) the 
trajectory of the system controlled by Prom lemmas 9, 10 and 11, there 
exists a sequence of strict controls such that 

dti^l' {da) = dt6u? {da) — > dt/it {da) Stably, V-a.s, 



E 



sup IVt-Vtl'^ 
te[o,T] 

T 



0, 



Jo 



E I \z?-zi'\'^dt — ^ 0. 



where (y", z") is the solution of equation (25) controlled by /x". 

According to the optimality of /i and (29), there exists a sequence (£n)„ 
of positive real numbers with lim Sn — such that 

J{u^')=J{n<J{l^)+en- 

A suitable version of lemma 13 implies that, given any > 0, there exists 
{vr-)n e U such that 

J {vJ") < inf J {u) + £n, 

J (m") <J{u) + End («", m) ; Vm e U. (37) 



24 



Let us define the perturbation 



,n,e 
't 



V ifte[T,T + 9], 
u? Otherwise. 



(38) 



Prom (37) we have 



0< Jiu 



) 



J{u")+Snd (u"'^0 



Prom the definition of the metric d, we obtain 



< J U- 



(39) 



From these above inequahties, we shall establish necessary optimality 
conditions for near optimal controls. 

Theorem 15 (Necessary optimality conditions for near controls). For each 
En > 0; there exists {u^)^ £ U such that there exists a unique adapted pro- 
cesses 



Proof. Prom inequality (39), we use the same method as in the last sections 
with index n. ■ 

5.3 Necessary and sufficient optimality conditions for 
relaxed controls 

In this subsection, we will state and prove necessary as well as sufficient 
optimality conditions for relaxed controls. Por this end, let us summarize 
and prove some of lemmas that we will use thereafter. 




(40) 



such that for all v G U, 



< [H {t, yr, z^,p7, <) - H {t, z^,pl v)] + Ce, 



(41) 
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(42) 



Introduce the following adjoint equation in the relaxed form 
r -dpf = {t, yl pI f,t) dt + {t, yi^, , pf, dWt, 

\ Po^9y (yo ) , 

where the Hamiltonian in the relaxed form is defined from [0, T] x x 
Mnxd (K) X R" X P (U) into R by 

H'' {t, yt, z^,p'^, Ht) =Pt b (t, y^, z^, a) jit (a)- h {t, y^, z^, a) jit (a) . 

Ju Ju 

For simplicity of notation, we denote 

nt)= I (t,yr,;^r, a) 
Ju 

where / stands for one of the functions by,bz,hy,hz- 



Lemma 16 The following estimations hold 

limE / |&"(s)-6^(s)fci5 = 0, 



limE / -6^(s)|^ds = 0, 

limE / \h:^{s)-h'^{s)fds^O, 
Jo 

limE / -/i^(s)|^cis = 0. 



(43) 
(44) 
(45) 
(46) 



Proof. We have 



E f\b;{s)-b>;{s)\'ds^E [ 
Jo Jo 



by {s, ys, z^, <) - / by {s, y^, z^, a) /x, (a) 
Ju 

<E f \by{s,y:,z:,u:)-by{s,y^,z:,u:)\'ds 

Jo 

+ E f \by {s, yl z:, u:) - by {s, yl z^, <) f ds 
Jo 



ds 



+ E 



by{s,y^,z^,u'',) - I by{s,y^,z^,a)i^s{a) 
u 



ds. 
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Since by is Lipschitz continuous in z, then 



E \by («) -b';{s)\'ds<E f \by {s, yl, z^,, <) - by (s, t/^ z^,, <)|^ ds 
Jo Jo 



+CE / \z':-z':rds 



(47) 



+ E 



^2/ (S, Vs, O - / ^2/ Vs, ^s, «) /^s 



(a) 



(is. 



Prom (32), we have 



hmE / \z^ - z^\^ ds ^ 0. 



Since by is bounded and continuous, then by (31) and the dominate con- 
vergence theorem, we have 

hm E /* \by {s, y:, z:, <) - by {s, y^, z^, <) f ds = 0. 

On the other hand, by the chattering lemma and the dominate conver- 
gence theorem, we have 



hmE 

n— »oo 



(s> a) ^< - / Ki.s,y'^,z^,a)ns{a) 

Ju 



ds = 0. 



By (47) and these above three hmits, we deduce (43). Using the same 
method and arguments, we prove (44) , (45) and (46). ■ 

Lemma 17 Let p"" and respectively the solutions of (40) and (42), then 
we have 



hmE 

n— >oo 



in A* 1 2 

sup \pt -p'^l 

«G[0,T] 



0. 



Proof. From (40) and (42), we have 

Pt = 9y iVo) - f {s) ds - f {s) dW^, 
Jo Jo 

Pr = 9y (y^) - f («) ds - f {s) dWs, 
Jo Jo 
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where 



Ju 

H: it) = {t, y^, zl\p^, <) ; (t) = [ {t, |/f , z^,p^, a) fit (a) . 

Ju 



Then 



E br -Ptf< CE \gy (y^) - (y^) f + CE f \H1^ {s) - (s) f ds 

Jo 

+ CE [ \H^ (s) - {s)f ds 
Jo 

<Ce[ \b; {s) {p: - p^,)\' ds + CE f \b:{s){p:-p^)fds + Ca^, 
Jo Jo 



where 



c^t=^\9y{yo)-9y{y^)\" + ^ f \K (s) - K {s)\' ds (49) 

+ E r I (6^ is) - 6^ is)) p^\' ds + E f \h: is) - K {s)f ds 
Jo Jo 

+ E fm{s)-b^As))p'sfds. 
Jo 



Since by and bz are bounded then 



E - p';f < 2CE f - p^l^ ds + Ca'i. 
Jo 



(50) 



Let us prove that hm a" = 

n— >oo 



Since Qy is bounded and continuous, then by (31) and the dominated 
convergence theorem, we have 

\\mE\gy{y^,)-gy{y^^)\'^Q. (51) 

n— »oo 

On the other hand, since by is bounded, then 

\[b''y{s)-b>;{s)\p:\<2C\p:\. (52) 
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Hence by the Cauchy-Schwartz inequality we get, 

Since e ([0, T] ; R"), then 

E \[ I K («) - («)] P^l < C (^E ^* {s) - 6^ (s) I' ' . 
By (43), we have 

hmE / \m{s)-hi'^{s)?ds^Q. 
Jo 

Then, we deduce that 

hm E / I [6" (s) - c^s = 0. (53) 

Jo 

By using the dominated convergence theorem we obtain 

hm E / I [6" (s) - 6(; (s)l pd' rfs = 0. (54) 

Similarly, using (44), the boundeness of and the dominated convergence 
theorem, it follows that 

hmE f\[b':{s)-b^A^)]p's\'ds = 0. (55) 

Prom (45) , (46) , (51) , (54) and (55), it is easy to see that 

lim = 0. (56) 



n— >oo 



Finally from (50) , (56), Gronwall's lemma and Bukholder-Davis-Gundy 
inequality, we have the desired result. ■ 
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Theorem 18 (Necessary optimality conditions for relaxed controls). Let /j, 
be an optimal relaxed control minimizing the cost J over TZ and {y^, z^) 
the corresponding optimal trajectory. Then there exists a unique adapted 
processes 

solution of the stochastic forward differential equation (42), such that for all 
q we have 

(t, yf, /X,) = me.xH^ (t, |/f , z>;,p^, q) . (57) 

geW{U) 

Proof. Let // be an optimal relaxed control. By the necessary condition for 
near controls (Theorem 15), there exists a sequence (ti"')„ C U such that for 
all V & U 

< [H {t, yr, z^,pl <) - H {t, yr, z^,p7, v)] + Csn, 
where lim = 0. 

n— >oo 

According to (29) , (31) , (31) and (48), the result follows immediately by 
letting n going to infinity in the last inequality ■ 

Remark 19 If fit (da) = 6u{t) {da), we recover the strict necessary optimality 
conditions (Theorem 4)- 

Theorem 20 (Sufficient optimality conditions for relaxed controls). We 
know that the setTZ of relaxed controls is convex and the function H'^ (t, y1, z^jpf, 
is linear in qt . If we assume that for every q ^ IZ and for all t e [0, T], the 
functions g is convex and {yl^z^) — > H'^ {t,yf, Zf,Pf,qt) is concave, then /i 
is an optimal relaxed control if it satisfies (57) . 

Proof. The proof is the same that in theorem 5. ■ 
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